1. Field of the Disclosure
This application relates generally to devices and methods to provide tactile sensory information from robotic or prosthetic finger tips comparable to the tactile sensing provided by human skin.
2. General Background and State of the Art
Present generations of robots lack most of the tactile sensorial abilities of humans. This limitation prevents industrial robots from being used to carry on delicate tasks of enormous practical relevance (such as assembly operations and handling of fragile objects) and, even more, it prevents the development of next-generation robots for off-factory jobs (agriculture, home, assistance to the disabled, etc.). Future generations of robots will need to make use of a wide variety of sensors and perceptual algorithms to identify and interact with objects and surfaces in the external world, particularly in environment that are less structured than those in which industrial robots are used now. Taction, vision, and proximity are the sensory needs that, in combination or alone, are commonly accepted as desirable features of robots. Research on visual pattern recognition received considerable attention in recent years. Tactile recognition (the ability to recognize objects by manipulation) is an inherently active process. Unlike visual sensors (passive and located remotely from the object), tactile sensors can be put in contact with the object to extract information about and, even more, such contact should be competently organized in order to extract the maximum degree of information from manipulative acts.
Humans who have suffered amputations of their hands and arms are generally provided with prosthetic limbs. Increasingly these prosthetics incorporate electromechanical actuators to operate articulations similar to biological joints, particularly to control the fingers to grasp and hold objects. Recent research has revealed how arrays of biological tactile receptors distributed throughout the soft tissues of the human finger tip are used normally by the nervous system to provide rapid adjustments of grip force when incipient slip is detected. Due to limitations in currently available tactile sensing technology discussed below, currently available prosthetic fingers provide little or no sensing capabilities and cannot make use of these highly effective biological control strategies.
Engineered tactile sensors detecting mechanical stimuli can be grouped into a number of different categories depending upon their construction. The most common groups are piezoresistive, piezoelectric, capacitive and elastoresistive structures. The common feature of all of these devices is the transduction of mechanical strains or deformations into electrical signals. Tactile sensors are commonly used in the field of robotics and in particular with those robotic devices that pick up and place objects in accordance with programmed instructions; the so-called “pick and place” class of robot. Unfortunately, while it would be desirable for the above-listed groups of tactile sensors to respond in much the same way that the human finger does, many of them can provide only limited information about contact with an object whose position, orientation and mechanical properties are highly predictable. More generalized sensing requires a multiplicity of sensors and extensive electrical connections and signal processing circuitry. It is difficult to integrate these components into the tactile surfaces of manipulators, which are often required to have contoured, compliant surfaces to facilitate handling of various objects. In order to achieve the requisite sensitivity, the individual sensors tend to be relatively fragile and subject to mechanical damage over the wide dynamic range of forces to which they may be exposed. The large number of electrical connections between sensors and signal processing circuitry tend to be difficult and expensive to assemble, difficult to protect from environmental hazards such as water and grit, and difficult or impossible to repair if damaged.
A wide variety of technologies have been applied to solve the tactile sensing problem in robotics and medicine. Transduction mechanisms such as optics, capacitance, piezoresistance, piezoelectricity, ultrasound, conductive polymers, etc. have all yielded viable solutions for detecting either normal pressure distributions, shear forces, or dynamic friction-induced vibrations but have required sensitive and fragile transducers to reside close to the contact surface to accurately detect these events. For example, most micro-electromechanical system (“MEMS”) sensors provide good resolution and sensitivity, but lack the robustness for many applications outside the laboratory.
Sensing of friction-induced vibrations has been a particular challenge in the development of tactile sensors. These vibrations arise when a compliant sensor is stroked across a surface at some velocity. When this occurs, the power transferred into the skin by friction gives rise to acoustic vibrations in the skin and pulp of the finger. The biological finger takes advantage of this phenomenon and has specialized sensors to detect these vibrations, which play an important role in slip-detection for reflexive grip-control. Many attempts to develop a sensor capable of measuring such small vibrations have been made (Howe, Cutkosky, Dario), but they have required fragile dynamic sensors residing very close to the contact surface to achieve the needed sensitivity. In this location fragile sensing devices are at a high risk for damage and experience short lifetimes and expensive repair costs.
The curved, deformable nature of biological finger tips provides mechanical features that are important for the manipulation of the wide variety of objects encountered naturally. Many tactile sensing arrays have been fabricated using MEMS but they are not suitable for mounting on such surfaces or for use in environments that include heavy loads, dust, fluids, sharp edges and wide temperature swings. If skin-like elastic coverings are placed on top of sensor arrays, they generally desensitize the sensors and function as low-pass temporal and spatial filters with respect to incident stimuli, thereby attenuating dynamic information.
It is a general property of biological sensory receptors that they are highly evolved structures in which the receptors themselves and the tissues in which they are located may contain many features designed to enhance their sensitivity and the quantity of information that they can provide to the central nervous system. The skin contains multiple types of mechanoreceptors to transduce a variety of mechanical events that occur during contact with physical objects. These receptors are concentrated in sites such as the finger tips, where their sensitivity is enhanced by the mechanical properties of the skin, underlying pulp and bone, and adjacent fingernails.
The input-output properties of these biological transducers differ generally from engineered transducers. Engineered transducers are usually designed to produce a linear response to a single mechanical variable such as normal or tangential force at a single point. The signals from arrays of such transducers can be combined according to simple, analytical algorithms to extract orthogonal physical parameters of touch such as total force, center of force, directional force vector and two-point resolution. Biological touch receptors are highly nonlinear and non-orthogonal. Their signals are combined by adaptive neural networks to provide subconscious adjustment of motor output as well as high level conscious perception associated with haptic identification of objects. Neurophysiologists and psychologists often correlate the activity of somatosensory receptors and design measures of psychophysical percepts according to canonical physical parameters, but there is little evidence that the nervous system actually extracts direct representations of such parameters as an intermediate stage between sensation and performance. In fact, information theory suggests that such an intermediate representation would add noise and reduce information content, which would place such a strategy at an evolutionary disadvantage.
Engineered sensors and their signal processing systems use linear, orthogonal representations because the downstream control systems generally have been based on such inputs. This strategy may work well for engineered systems such as industrial robots that can perform accurately for highly constrained and predictable tasks. It is difficult to apply to anthropomorphic robots and prosthetic limbs that can perform a broad and unpredictable range of tasks associated with activities of daily living. The problem may further be complicated by environmental factors in such environments (e.g. temperature, moisture, sharp edges etc.), which tend to damage or bias sensitive and/or physically exposed transducers.
U.S. Pat. No. 4,980,646, to Zemel (“Zemel”), is incorporated in its entirety herein by reference and teaches a tactile sensor based on changes in the local electrical resistance presented by a layer of weakly conductive fluid whose shape is deformed by external forces applied to a deformable membrane. Zemel describes the application of a voltage gradient across the entire extent of the fluid by means of electrodes arranged on either side of the array of sensing strips, and the measurement of the local strength of that gradient by differential voltage measurements between adjacent pairs of electrode strips. U.S. Pat. No. 4,555,953 to Dario et al., which is incorporated herein by reference in its entirety, teaches different techniques and materials that have been utilized for the construction of artificial skin-like sensors.
The following articles are referred to throughout the disclosure and their contents are incorporated by reference herein in their entireties: Lee M. H., Nichols H. R., Tactile sensing for mechatronics—a state of the art survey, Mechatronics 9:1-31 1999. Beccai L., Design and fabrication of a hybrid silicon three-axial force sensor for biomechanical applications Sensors and Actuators, A. Physical. Vol. A120, no. 2: 370-382. 17 May 2005. Mei T., et al., An integrated MEMS three-dimensional tactile sensor with large force range, Sensor and Actuators 80:155-162, 2000. Beebe D., et al., A silicon force sensor for robotics and medicine, Sensors and Actuators A 50:55-65, 1995. Bloor D., et al., A metal-polymer composite with unusual properties, Journal of Physics D: Applied Physics, 38: 2851-2860, 2005. Vasarhelyi G., et al. Effects of the elastic cover on tactile sensor arrays. Sensors and Actuators 132:245-251, 2006. Helsel, M., et al., An impedance tomographic tactile sensor, Sensor and Actuators. Vol. 14, No. 1, pp. 93-98. 1988. Russell, R. A., Parkinson, S., Sensing surface shape by touch, IEEE International Conference on Robotics and Automation, Vol. 1 423-428, 1993. Kenaly G., Cutkosky M., Electrorheological fluid-based robotic fingers with tactile sensing, Proceedings of IEEE International Conference on Robotics and Automation 1:132-136, 1989. Voyles R., et al., Design of a modular tactile sensor and actuator based on an electrorheological gel, Proceedings of IEEE International Conference on Robotics and Automation, 1:132-136, 1989. Lee Y. K., et al., Mechanical properties of calcium phosphate based dental filling and regeneration materials, Journal of Oral Rehabilitation 30; 418-425, 2003. D. Merrill, et al., Electrical stimulation of excitable tissue: design of efficacious and safe protocols, Journal of Neuroscience Methods, 141: 171-198, 2005. A. Dalmia, et al., Electrochemical behavior of gold electrodes modified with self-assembled monolayers with an acidic end group for selective detection of dopamine, Journal of Electrochemistry, 430: 205-214, 1997. B. Piela, P. Wrona, Capacitance of the gold electrode in 0.5 M sulfuric acid solution: AC impedance studies, Journal of Electrochemistry, 388: 69-79, 1994. Johansson R., et al., Somatosensory control of precision grip during unpredictable pulling loads, Changes in load force amplitude, Experimental Brain Research 89: 181-191, 1992. Birznieks I., et al, Encoding of direction of fingertip forces by human tactile afferents, Journal of Neuroscience. 21:8222-8237, 2001. Flanagan J. R., et al. Control of fingertip forces in multi-digit manipulation, Journal of Neurophysiology. 81:1706-1717, 1999. Johansson R. S., Westling G., Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects, Experimental Brain Research. 56:550-564, 1984. Johansson R. S., Westling G., Signals in tactile afferents from the fingers eliciting adaptive motor responses during precision grip, Experimental Brain Research, 66:141-154, 1987. Westling G., Johansson R. S., Responses in glabrous skin mechanoreceptors during precision grip in humans, Experimental Brain Research. 66:128-140, 1987. K. Hornik, et al., Multilayer feed forward networks are universal approximators, Neural Networks, 2(5):359-366, 1989. Park, J. and I. Sandberg, Approximation and radial-basis-function networks, Neural Computation 5, 305-316, 1993. Caudill, M.; Butler, C., Understanding Neural Networks: Computer Explorations; Volume 1: Basic Networks; The MIT Press; Cambridge, Mass., 1992. D. Yamada, et al., Artificial Finger Skin having ridges and distributed tactile sensors used for grasp force control, Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 686-691, 2001. Y. Mukaibo, et al., Development of a texture sensor emulating the tissue structure and perceptual mechanism of human fingers, Proc. of the 2005 IEEE International Conference on Robotics and Automation, pp. 2576-2581, 2005. Johansson R. S. and Westling G., Role of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher and more slippery objects, Experimental Brain Research 56: 550-564, 1984. Cole K. J., Johansson R., Friction at the digit-object interface scales the sensory-motor transformation for grip responses to pulling loads, Experimental Brain Research, 95: 523-532, 1993. Johansson R., et al., Somatosensory control of precision grip during unpredictable pulling loads, II Changes in load force rate, Experimental Brain Research 89: 192-203, 1992. Gordon A., et al., Memory representation underlying motor commands used during manipulation of common and novel objects, Journal of Neurophysiology 69: 1789-1796, 1993. Johansson R. S., Birznieks I., First spikes in ensembles of human tactile afferents code complex spatial fingertip events, Nature Neuroscience 7:170-177, 2004. Butterfass, J., DLR-Hand II: Next generation of a dexterous robot hand, Proc. of the 2001 IEEE, International Conference on Robotics & Automation, Seoul, Korea, May 21-26, 2001. Mountcastle V. B., The view from within: Pathways to the study of perception, The John Hopkins Medical Journal, 136:109-131, 1975. Wettels N., et al., Biomimetic Tactile Sensor for Control of Grip, IEEE Rehabilitation Robotics, 2007, Proceedings of the IEEE International Conference on Robotics and Automation, pp 109-114, 2001. N. Wettels, et al., “Biomimetic tactile sensor array” Advanced Robotics, vol. 22, no. 7, June 2008.